Model-Based Policy Iterations for Nonlinear Systems via Controlled Hamiltonian Dynamics

The infinite-horizon optimal control problem for nonlinear systems is studied. In the context of model-based, iterative learning strategies we propose an alternative definition and construction temporal difference error arising in Policy Iteration strategies. such architectures error computed via evolution Hamiltonian function (or, possibly, its integral) along trajectories closed-loop system. Herein temporal instead obtained two subsequent steps: first dynamics underlying costate variable system steered by means a (virtual) input way that stable invariant manifold becomes externally attractive. Then, distance-from-invariance manifold, induced approximate solutions, yields natural candidate measure policy evaluation step. improvement phase then performed standard gradient descent methods allows to correctly update weights functional approximator. above architecture iterative (episodic) learning scheme based on scalar, constant reward at each iteration, value which insensitive length episode, as original spirit Reinforcement Learning discrete-time systems. Finally, theory validated numerical simulation involving automatic flight problem.